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ABSTRACT 

In  this  paper,  we  suggests  how  to  segment  the  face  when  there  is  the  man  under  complex  environment,  extracts  the  features 
from  segmented  the  image  and  proposes  a effective  recognition  system  using  the  discrete  wavelet  transform  (DWT).  This 
algorithm  is  proposed  by  following  processes.  First,  two  gray-level  images  is  captured  with  256  level  of  the  size  of  256X 
256  in  constant  illumination.  We  use  a Gaussian  filter  to  remove  noise  of  input  image  and  get  a differential  image  between 
background  and  input  image.  Second,  a mask  is  made  from  erosion  and  dilation  process  after  binary  of  the  differential 
image.  Third,  facial  image  is  divided  by  projecting  the  mask  into  input  image.  Most  characteristic  information  of  human 
face  is  in  eyebrow,  eyes,  nose  and  mouth.  In  the  reason,  the  facial  characteristic  are  detected  after  selecting  local  area 
including  this  area.  Forth,  detecting  the  characteristic  of  segmented  face  image,  edge  is  detected  with  Sobel  operator.  Then, 
eye  area  and  the  center  of  face  are  searched  by  using  horizontal  and  vertical  components  of  edge.  Characteristic  area 
consists  of  eyes,  a nose,  a mouth,  eye  brows  and  cheeks,  is  detected  by  searching  the  edge  of  the  image.  Finally, 
characteristic  vectors  are  extracted  from  performing  DWT  of  this  characteristic  area  and  are  normalized  it  between  +1  and  - 
1.  Normalized  vectors  is  used  with  input  vector  of  neural  network.  Simulation  results  show  recognition  rate  of  100  % about 
learned  image  and  92%  about  test  image. 

Keyword  : Mask,  Differential  image,  Discrete  wavelet  transform,  Neural  network,  Face  recognition 

1.  INTRODUCTION 

Edge  image  is  mainly  used  to  discriminate  whether  there  is  face  or  not  in  the  image.  Sakai  applied  oval  mask  to  edge  map 
extracted  from  input  image,  set  the  approximate  head  area,  checked  the  edge  image  of  eyes  and  mouth  within  the  head  area 
and  then  extracted  the  final  head  area*1!  This  method  has  the  weakness  that  it  is  influenced  greatly  by  the  direction  of 
lighting.  Kelly  also  produced  the  downward  image  interpretation  method  that  extracts  the  outline  of  head  and  body 
automatically  from  input  image  and  continuously  extract  the  location  of  eyes,  nose  and  mouth*2!  Craw  and  others 
proposed  the  method  extracting  head  area  using  mask  of  hierarchical  size  in  a given  image*3!  Using  the  outline  of  head 
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which  is  composed  of  edge  image,  Govindaraju  searched  the  face  in  the  complex  background  image[4].  Sirohey  segmented 
the  face  by  using  edge  image  and  brightness  image  which  are  extracted  with  Canny  edge  searcher  in  the  image  with 
background^.  This  method  showed  the  precision  of  about  80%  about  48  images  without  any  restriction.  In  case  of 
recognizing  segmented  images,  feature  extraction  methods  consists  of  local  feature  extraction  and  global  feature  extraction 
algorithm. [6^  Feature  location  detection  methods  include  the  method  by  geometrical  symmetry,  method  using  correlation  of 
feature  templates  like  eyes,  nose  and  mouth  and  image,  method  detecting  face  to  search  face  candidate  edges  with  snakelets, 
method  detecting  features  of  face  with  self-organizing  feature  map  and  method  extracting  the  feature  from  frequency 
domain  like  FFT,  DCT.  After  extracting  the  features,  generally,  Euclidean  distance  and  neural  network  is  used  to  recognize 
the  face.  While  the  former  has  the  strength  that  system  implementation  is  easy,  it  has  also  weakness  that  recognition  rate  is 
lowered  when  there  are  a lot  of  data.  In  case  of  performing  pattern  recognition  with  the  latter,  Input  images  are  converted 
into  spatial  or  frequency  domain.  Then,  feature  parameters  of  the  image  are  extracted  and  it  is  used  with  input  vectors  of 
neural  network.  This  method  has  the  properties  that  the  number  of  nodes  and  connection  lines  can  be  reduced  and  system 
implementation  are  easyt7l  In  this  paper,  we  propose  that  use  differential  image  to  obtain  a mask  and  segment  the  face  using 
a mask.  Also,  we  define  the  characteristic  areas  of  face  to  reduce  the  computational  complexity  in  extracting  the 
characteristic.  And  we  extract  the  exact  characteristic  vectors  using  discrete  wavelet  conversion.  This  method  reduces  the 
number  of  characteristic  vectors  and  decreases  the  required  learning  data  in  learning  with  neural  network.  In  section  2,  we 
describe  face  segmentation  algorithm  based  on  differential  image.  Section  3 describes  face  recognition  algorithm  based  on 
wavelet.  Section  4 describe  simulation  results.  Conclusions  are  given  in  section  5. 

2.  FACE  SEGMENTATION  ALGORITHM  BASED  ON  DIFFERENTIAL  IMAGE 

2.1  Face  Segmentation  Using  Differential  Image  and  Mask 

The  face  is  segmented  by  applying  differential  images  in  regular  illumination  condition.  Image  is  obtained  with  gray  scale 
256  level  of  the  size  of  256X256  and  the  noise  in  the  image  is  rejected  by  Gaussian  filtering.  And  since  the  pixel  value 
outside  the  face  in  input  image  including  the  face  is  not  identical  with  that  of  background  image  which  doesn't  include  the 
face  exactly  in  obtaining  the  difference  of  input  images  including  background  image  and  face,  threshold  is  given  as  shown 
in  the  following  Expression  (1)  and  the  pixel  value  of  the  image  obtained  with  the  same  camera  can  get  the  desired  image 
though  it  is  changed  a little. 

If  | Imagel(x,y)  - Image  2(x,y)  \ < threshold 
Then  Differ _Im  age(x.y)  — 0 

Else  Differ _Image(x,y)  = \Imagel  (s,y)  - Image2(x,y)\  (1) 

For,  Image  1,  Image2  : input  image,  Differ  Jmage  : differential  image 
The  noise  occurred  by  the  change  of  illumination  and  the  reflection  of  light  occupies  small  area  in  the  image  because  of  low 
frequency  of  noise  and  it  makes  binary  differential  image  and  erodes  the  image.  Eroded  binary  image  should  be  dilated  to 
the  size  of  face  image.  Then  pixel  value  is  examined,  mask  is  made,  it  is  projected  on  the  original  image  with  the  face  and 


the  face  is  segmented  in  the  background.  Fig.  1 shows  the  flow  chart  to  segment  the  face. 


2.2  Feature  Detection  of  Face 

Most  feature  information  of  human  face  is  in  eyebrow,  eyes,  nose,  mouth  and  cheek.  So  the  face  features  are  detected  after 
selecting  local  area  including  this  area.  For  the  feature  detection  of  segmented  face  image,  edge  is  detected  with  sobel 
operator  and  eye  area  and  the  center  of  face  are  searched  by  using  horizontal  and  vertical  components  of  edge. 


Fig.  1 Flow  chart  for  facial  image  segmentation 


Further  since  the  size  of  human  faces  in  regular  distance  is  similar,  square  area  should  be  defined  to  include  eyebrow,  eyes, 
nose  and  cheek  like  Fig.  2 and  it  is  considered  as  the  feature  area  of  face  recognition.  After  the  conversion  of  discrete 
wavelet  of  extracted  feature  areas,  extract  the  features.  a(a=c+e+f)  in  Fig.  3 indicates  horizontal  distance  of  characteristic 
area  and  b the  vertical  distance. 


Fig.  2 Flow  chart  for  characteristic  detection 
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Fig.  3.  The  normalized  distance  for  characteristic  detection. 

3.  FACE  RECOGNITION  ALGORITHM  BASED  ON  WAVELET  TRANSFORM 

This  chapter  proposes  the  method  of  recognizing  the  face  image  by  using  neural  network  after  extracting  the  characteristics 
of  face  in  wavelet  conversion  area.  Fig.  4 shows  the  algorithm  flow  chart  of  face  recognition.  Wavelet  is  obtained  by 
dilate/translating  mother  wavelet  \|/(x)  and  it  is  shown  in  Expression  (2). 

(2) 

where,  a is  scaling  parameter,  b translation  parameter  and  V a normalization  factor.  Wavelet  conversion  Wf(s)  of  random 
signal  f(x)  is  defined  as  convolution  integration  with  original  signal  f(x)  and  wavelet  (x)  as  shown  in  Expression  (3). 

Wf(x)  = f*(p(x)  (3) 

In  addition,  when  ^ (x,y)  = ig  (x)xg  (y)  is  two-dimensional  scaling  function  in  wavelet  conversion  of  two-dimensional 
function  f(x,  y)  and  one-dimensional  wavelet  with  one-dimensional  scaling  function  tg  (x)  is  ^ (x),  two-dimensional 
wavelets  can  be  explained  to  separable  multi-resolution  approximation  as  shown  in  Expression  (4). 

(pA(x,y)  = tp(x)tp(y) 

<p'(x,y)=<l>(x)<p(y) 

<p“(x,y)  = <Kx)<Ky)  (4) 

f°(x,y)  = <P(x)<P  O') 
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Fig.  4 Flow  chart  for  face  recognition  algorithm 

Two-dimensional  signal  divided  into  normal  orthogonal  distance  of  Expression  (4)  is  divide  into  frequency  component  of 
spatial  direction.  The  extraction  process  of  feature  parameters  using  wavelet  conversion  d;  ides  two  input  image  signals 
with  resolution  of  [256X256X28]  into  characteristic  areas  with  pixel  of  91X91  using  differential  image  and  edge  feature. 
Then  DWT  coefficient  matrix  is  obtained  with  DWT.  The  following  Fig.  5 shows  the  distribution  of  coefficient  matrix  of 
4 level  DWT,  where  cA4  means  coefficient  matrix  of  4 level  low  frequency,  cH(i)  horizon  al  high  frequency  coefficient 
matrix  of  (i)  level,  cV(i)  vertical  high  frequency  coefficient  matrix  of  (i)  level  and  cD(i)  diagonal  high  frequency  coefficient 
matrix  of  (i)  level.  Feature  vectors  are  extracted  by  using  such  coefficient  matrixes.  When  our  level  DWT  of  segmented 
facial  characteristic  areas  is  performed,  cA4,  cH4,  cV4  and  cD4  which  are  coefficient  matri  ^es  with  the  size  of  6X6  are 
obtained,  where  cA4  is  four  level  low  frequency  coefficient  matrix,  cH4  horizontal  high  frequency  coefficient  matrix,  cV4 
vertical  high  frequency  coefficient  matrix  and  cD4  diagonal  high  frequency  coefficient  matrix,  and  after  analyzing  the 
distribution  features  of  these  matrixes,  extract  feature  vectors.  First  to  examine  the  distribution  of  coefficient  matrix  of  the 
same  person's  learning  image,  four  level  DWT  is  performed  with  four  sample  learning  ; mages.  Then  get  the  absolute 
values  of  coefficients  of  four  coefficient  matrixes  with  the  size  of  6X6,  extract  36  feature  vectors  and  normalize  them 
between  +1  and  -1.  Finally  after  calculating  mean  square  error  of  normalization  vector  extracted  from  cA4,  cH4,  cV4  and 
cD4,  utilize  it  as  learning  vector  of  neural  network  based  on  the  size  of  error. 
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Fig.  5 Four  Level  DWT  Coefficient  Matrix 

4.  SIMULATION 


4.1  Face  Segmentation  and  characteristic  Detection 

This  study  acquires  the  input  image  in  the  same  distance  from  CCD  camera  at  regular  illumination  condition  with  gray  scale 
256  level  of  256X256  and  simulation  is  performed  on  PC(266MHz).  For  this,  the  noise  of  input  image  like  Fig.  6 and  7 is 
rejected  by  using  gaussian  filter,  differential  image  is  obtained  and  then  the  face  is  segmented  in  background  image, 


Fig.  6 Input  Image(I)  Fig.  7 Input  Image  (II) 

Then  the  differential  image  of  two  input  images  is  obtained.  Fig.  8 shows  the  resulting  differential  image  and  Fig.  9 is 
binary  image. 


Fig.  8 Differential  Image  Fig.  9 Binary  Image 

Since  the  differential  image  cannot  be  immediately  used  as  mask,  it  is  binary  as  shown  in  Fig.  9 and  the  boundary  is  reduced. 
But  since  the  reduction  of  boundary  results  in  that  of  face  area,  the  eroded(reduced)  image  is  dilated  as  shown  in  Fig.  10. 
Fig.  1 1 is  mask  image  generated  by  examining  the  pixel  value  from  dilated  image. 
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Fig.  10  Dilated  Image  Fig.  1 1 Mask  Image 

And  mask  image  is  projected  on  original  image  with  face  and  the  face  is  segmented  in  the  background  as  shown  in  Fig.  12. 


Fig.  12  Segmentation  Image  Fig.  13  Edge  Image 

After  detecting  edge  using  sobel  operator  in  segmented  face  image,  detect  the  position  of  eyes  and  eyebrow  by  horizontal 
distribution  of  edge  components  and  segment  the  lower  part  of  eyebrow.  Then  get  the  vertical  distribution  of  edge  and  the 
center  line  of  face.  Finally,  the  characteristic  areas  are  detected  on  the  basis  of  the  knowledge  of  human  face  as  shown  in 
Fig.  14. 


Fig.  14  Facial  Characteristic  Area 


4.2  Face  Recognition  Experiment 

Fig.  1 5(a)  — 1 5(d)  are  characteristic  area  separated  from  four  experimental  images  of  the  same  figure  and  their  size  is  91  x 9 1 
respectively. 
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Fig.  15  Example  of  Separated  Characteristic  Image 

Fig.  16  shows  the  image  of  coefficient  matrixes  of  four  level  wavelet  conversion  cA4,  cH4,  cV4  and  cD4  and  the 
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information  of  original  image  is  concentrated  on  these  four  coefficient  matrixes.  The  size  of  these  coefficient  matrixes  is  6 
X6  respectively.  For  feature  extraction,  four  learning  samples  of  the  image  in  Fig.  15(a) — 15(d)  are  selected,  they  are 
converted  into  four  level  wavelet  and  then  cA4,  cH4,  cV4  and  cD4  are  obtained.  Then  after  analyzing  normalization  data 
distribution  of  the  same  coefficient  matrixes,  extract  characteristic  vector  and  it  is  the  input  vector  of  neural  network  by 
normalizing  it  between  +1  and  -1.  Fig.  17  shows  the  distribution  condition  of  normalized  characteristic  vector. 
Normalization  is  performed  after  taking  the  absolute  value  of  6X6  coefficient  matrix  obtained  from  four  sample  images, 
where  horizontal  direction  is  the  number  of  normalized  characteristic  vectors  (36)  and  vertical  direction  is  the  range  of 
normalization  between  +1  and  -1.  Error  among  normalized  vectors  of  the  same  coefficient  matrix  is  obtained  to  select 
input  vector  of  neural  network  of  coefficient  matrixes  from  four  sample  images  like  figure  15(a)  — 15(d).  First,  RMSE  of 
normalized  characteristic  vector  is  obtained  in  cA4.  Then  RMSE  of  cH4,  cV4  and  cD4  is  obtained  through  the  same 
process.  The  following  Table  1 shows  RMSE  of  each  coefficients. 


Fig.  16  Four  Step  Decomposition  Image 

Total  108  of  each  36  from  cA4,  cH4  and  cV4  are  extracted  as  characteristic  vectors  based  on  Table  1 and  these  are  the  input 
vectors  of  neural  network.  Fig.  18  shows  the  distribution  of  108  neural  network  input  vector  extracted  from  four  sample 
images  of  the  same  person. 


Fig.  17  Normalized  Characteristic  Vectors 


Table  1 . RMSE  of  normalized  feature  vectors 


Class 

cA4 

cH4 

cV4 

cD4 

RMSE 

0.0035 

75 

0.00486 

7 

0.00638 

1 

0.00658 

0 

To  confirm  the  validity  of  characteristic  vector  extraction,  four  images  of  three  persons  are  obtained,  wavelet  conversion  is 
conducted  and  the  normalized  results  of  coefficient  matrix  are  shown.  For  this,  four  sample  images  of  experimental  image 
like  Fig.  19(a),  19(b)  and  19(c)  are  selected,  four  level  wavelet  conversion  of  theses  images  is  performed,  6X6  coefficient 
matrix  is  normalized  and  36  characteristic  vectors  are  extracted.  And  RMSE  among  the  same  coefficients  is  obtained. 


Fig.  18.  Extracted  Input  Vectors 

Table  2,  3 and  4 are  RMSE  of  normalized  vectors.  The  largest  error  is  found  in  normalized  vector  of  cD4. 


Table  2.  RMSE  of  normalized  feature  vectors 


Class 

cA4 

CH4 

cV4 

cD4 

RMSE 

0,0030 

76 

0.00556 

5 

0.00304 

2 

0.00672 

6 

Table  3.  RMSE  of  normalized  feature  vectors 


Class 

cA4 

CH4 

cV4 

cD4 

RMSE 

0.0017 

88 

0.00383 

0 

0.00219 

6 

0.00622 

4 

Table  4.  RMSE  of  normalized  feature  vectors 


class 

cA4 

cH4 

cV4 

CD4 

RMSE 

0.0008 

48 

0.00292 

8 

0.00108 

2 

0.00347 

6 
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Fig.  20(a),  20(b)  and  20(c)  shows  the  neural  network  input  vector  extracted  from  Fig.  19(a),  19(b)  and  19(c). 


(a)  (b)  (c) 

Fig.  19  Example  of  experimental  images. 

Such  108  normalized  input  vectors  are  multi-layer  neural  network.  Learning  algorithm  of  neural  network  uses  error 
Backpropagation  learning  algorithm.  The  error  of  output  layer  is  0.005  and  learning  rate  is  0.7.  Weight  of  network 
generated  after  learning  and  feature  vector  of  input  comparative  image  are  operated  and  the  recognition  is  performed  by 
comparing  the  error  of  output  layer.  Then  when  the  error  of  output  layer  is  less  than  0.005,  it  is  judged  as  the  same  person. 


4.3  Comparison  and  Examination 

The  simulation  results  of  proposed  algorithms  are  as  follows. 

1)  Facial  image  can  be  segmented  100%  in  proposed  experimental  environment  by  using  face  segmentation 
algorithm  based  on  differential  image. 

2)  More  elaborate  feature  vector  can  be  obtained  than  by  using  DCT  and  the  recognition  rate  increases  4%.  Fig.  21 
shows  that  the  normalization  characteristic  of  feature  vectors  extracted  after  wavelet  conversion  is  more  evenly  distributed 
than  the  distribution  characteristic  of  extracted  after  performing  DCT  of  the  same  person. 


(a)  In  case  of  DCT  (b)  In  case  of  4 level  DWT 

Fig.  21  Distribution  of  Characteristic  Vectors 

3)  Features  are  extracted  by  converting  the  data  of  spatial  area  into  frequency  area  and  data  quantity  can  be  reduced. 
Also  data  quantity  for  operation  is  reduced  by  handling  characteristic  area  not  the  whole  face  in  performing  DWT. 

4)  Since  the  wrong  recognition  of  included  face  image  occurs,  the  compensation  of  inclination  is  required.  And 
algorithm  such  as  pass  word  input  or  fingerprint  recognition  is  required  to  apply  it  to  perfect  complementary  system. 

5.  CONCLUSIONS 

We  have  presented  our  experiments  about  face  segmentation  method  based  on  differential  and  face  recognition  using 
discrete  wavelet  transform.  This  algorithm  was  proposed  by  following  processes.  First,  two  gray-level  images  was 
captured  with  256  level  of  the  size  of  256X256  in  constant  illumination.  W used  a Gaussian  filter  to  remove  noise  of  input 
image  and  got  a differential  image  between  background  and  input  image.  Second,  a mask  was  made  from  erosion  and 
dilation  process  after  binary  of  the  differential  image.  Third,  facial  image  was  divided  by  projecting  the  mask  into  input 
image.  Most  characteristic  information  of  human  face  was  in  eyebrow,  eyes,  nose  and  mouth.  In  the  reason,  the  facial 
characteristic  were  detected  after  selecting  local  area  including  this  area.  Forth,  detecting  the  characteristic  of  segmented 
face  image,  edge  was  detected  with  Sobel  operator.  Then,  eye  area  and  the  center  of  face  were  searched  by  using  horizontal 
and  vertical  components  of  edge.  Characteristic  area  consists  of  eyes,  a nose,  a mouth,  eye  brows  and  cheeks,  was 
detected  by  searching  the  edge  of  the  image.  Finally,  characteristic  vectors  were  extracted  from  performing  DWT  of  this 
characteristic  area  and  are  normalized  it  between  +1  and  -1.  Normalized  vectors  was  used  with  input  vector  of  neural 
network.  Through  simulation  results,  we  shown  recognition  rate  of  100  % about  learned  image  and  92%  about  test  image. 
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